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(57)Abstract: 

PURPOSE: To obtain a correction function for correcting distortion from image data. 
CONSTITUTION: A checkered characteristic pattern is picked up by an image input 
device 10 and feature points (intersection) are detected from the image pickup data. A 
mapping function obtained when a pinhole camera is used as a television camera is 
generated from an array of feature points nearby the center and the position on a 
screen. The reference position of the feature points as to the whole screen is 
calculated from the mapping function and compared with the actual position on the 
screen to calculate the correction function, which is stored in a correction function 
generating device 32. Then, the input image is normally corrected with the correction 
function and outputted to an image output device 20. 
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CLAIMS 



[Claim(s)] 

[Claim 1] The focus extract process of extracting the focus from the image pick-up 
data of the pattern for proofreading whose arrangement of the focus it is the 
distortion amendment approach in the image processing system which amends 
distortion of the image in the image pick-up data obtained with image pick-up 
equipment, and is known, The location of the focus of the predetermined number near 
the center in the image pick-up data obtained by this focus extract means, Based on 
the data about the pattern for proofreading, image pick-up equipment is assumed to 
be a pinhole camera. The mapping function calculation process which computes the 
mapping function which shows both relation, and the criteria location calculation 
process which computes the criteria-almost location in the image pick-up data of the 
focus all over the districts of image pick-up data using the mapping function obtained 



with this mapping function calculation means, The distortion amendment approach 
characterized by having the correction function calculation process which computes 
the correction function for amending the location of the focus based on the location 
of the focus obtained by this criteria location calculation means, and the location of 
the focus in image data, and using this correction function for distortion amendment 
of image pick-up data. 

[Claim 2] It is the distortion correction function calculation approach in the image 
processing system which amends distortion of the image in the image pick-up data 
obtained with image pick-up equipment. The focus extract process of having a 
checkered pattern and extracting an intersection location from the image pick-up 
data about the pattern for proofreading whose arrangement of the intersection in a 
pattern is known as the focus, The correction function calculation process which 
computes the correction function for amending the location of the focus based on the 
location of the focus obtained by this focus extract means, and the data about the 
pattern for proofreading, It is the distortion amendment approach which amends 
distortion of image pick-up data using the correction function obtained according to 
the implication and the correction function calculation process. The above-mentioned 
focus calculation process The evaluation value for every processing-object location is 
computed using the template which defines the operation means about the image 
pick-up data of the circumference including a processing-object location. The 
distortion amendment approach characterized by computing the boundary line of a 
pattern from the change condition of the image pick-up data of the circumference 
computed from this evaluation value, and computing the location of the focus from the 
intersection of this boundary line. 

[Claim 3] The correction function storage section which is the image-distortion 
compensator which amends distortion of an image from the image pick-up data 
obtained with image pick-up equipment, and memorizes the correction function for 
amending image pick-up data, An amendment means to amend image pick-up data 
using the correction function read from this correction function storage section is 
included. The above-mentioned correction function storage section From the image 
pick-up data of the pattern for proofreading whose arrangement of the focus is known, 
extract the focus and among the obtained focus The location of the thing near the 
center in image pick-up data, Based on the data about the pattern for proofreading, 
the mapping function which assumed image pick-up equipment to be a pinhole camera 
is computed. The image-distortion compensator mostly characterized by memorizing 
the correction function obtained based on the criteria location of the focus of image 
pick-up data which computed the criteria location in the image pick-up data of the 
focus all over the districts, and was obtained, and the location of the focus in image 
data using the obtained mapping function. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Industrial Application] This invention relates to the image processing which obtains 
the image which processes the image pick-up data obtained with image pick-up 
equipment, and does not have distortion. 
[0002] 

[Description of the Prior Art] Conventionally photoing an object object and a 
surrounding situation is performed by image pick-up equipments, such as a CCD 
camera, and processing the obtained image pick-up data and detecting an objective 
location is performed. Especially, in the intelligent robot who acts independently within 
a general environment while judging a situation himself, the vision of a three dimension 
is important, and to recognize an objective three-dimension location correctly from 
the image pick-up data obtained with image pick-up equipment is desired. 
[0003] Especially, as compared with the case where the camera tube till then is used, 
distortion of an image decreases sharply by use of a CCD camera, and an exact image 
without distortion is obtained increasingly. Then, it is coming to be able to perform 
location recognition of a quite exact body by processing of the obtained image pick-up 
data. That is, an objective location can be recognized by modeling the digital image 
using a CCD camera etc. as transparent transformation. 

[0004] However, the digital image obtained with the CCD camera also has distortion 
by various factors in fact, and a model is not wholly exact the bottom in transparent 
transformation. Then, conventionally, in case a three-dimension location is detected 
from image pick-up data, various kinds of amendments are performed about image 
pick-up data, and improving the precision of detection is proposed. 
[0005] for example, a scientific publication — " — the Information Processing Society 
of Japan research report [ ] — Vol.92 and No. — others [ / 7 p.1 15 - 1 18 January, 
1992:Onodera ] — " — **** — the grid pattern which was installed on monotonous 
and drawn is photoed with image pick-up equipment so that it may intersect 
perpendicularly to the optical axis of a camera, and amending image pick-up data is 
shown. That is, it aims at amending distortion resulting from the aberration of a lens, 
and this example defines a suitable correction function in consideration of the 



property of this distortion. And a correction function is asked for a multiplier based on 
the comparison of the grid pattern which the physical relationship in the inside of 
three-dimension space understands beforehand to be image pick-up data for every 
pixel, and the boundary of a grid pattern being straight lines. And image pick-up data 
are amended using this correction function. 
[0006] 

[Problem(s) to be Solved by the Invention] Thus, distortion of an image can be 
amended using the relation between the location in the inside of three-dimension 
space, and the location picturized in the image, and recognition of an objective 
location etc. can be made more exact. However, it is necessary to install the target 
grid pattern etc. in a known location to image pick-up equipment including an 
above-mentioned example by the conventional technique, and it difficult to install 
correctly by fluctuation of the optical axis of image pick-up equipment etc. in fact. 
Therefore, there was a trouble that it could not amend correctly, by this technique. 
[0007] This invention aims at obtaining the approach of amending an exact image 
distortion in view of the above-mentioned technical problem, and equipment. 
[0008] 

[Means for Solving the Problem] This invention is the distortion amendment approach 
in the image processing system which amends distortion of the image in the image 
pick-up data obtained with image pick-up equipment. The focus extract process of 
extracting the focus from the monotonous image pick-up data describing the pattern 
for proofreading whose arrangement of the focus is known, The location of the focus 
of the predetermined number near the center in the image pick-up data obtained by 
this focus extract means, Based on the data about the pattern for proofreading, image 
pick-up equipment is assumed to be a pinhole camera. The mapping function 
calculation process which computes the mapping function which shows both relation, 
and the criteria location calculation process which computes the criteria-almost 
location in the image pick-up data of the focus all over the districts of image pick-up 
data using the mapping function obtained with this mapping function calculation means, 
Based on the location of the focus obtained by this criteria location calculation means, 
and the location of the focus in image data, it has the correction function calculation 
process which computes the correction function for amending the location of the 
focus, and is characterized by using this correction function for distortion amendment 
of image pick-up data. 

[0009] Moreover, it is the distortion correction function calculation approach in the 
image processing system which amends distortion of the image in the image pick-up 
data obtained with image pick-up equipment. The focus extract process of having a 
checkered pattern and extracting an intersection location from the image pick-up 
data about the pattern for proofreading whose arrangement of the intersection in a 
pattern is known as the focus, The correction function calculation process which 



computes the correction function for amending the location of the focus based on the 
location of the focus obtained by this focus extract means, and the data about the 
pattern for proofreading, It is the distortion amendment approach which amends 
distortion of image pick-up data using the correction function obtained according to 
the implication and the correction function calculation process. The above-mentioned 
focus calculation process The evaluation value for every processing-object location is 
computed using the template which defines the operation means about the image 
pick-up data of the circumference including a processing-object location. It is 
characterized by computing the boundary line of a pattern from the change condition 
of the image pick-up data of the circumference computed from this evaluation value, 
and computing the focus from the intersection of this boundary line. 
[0010] Moreover, the correction function storage section which is the 
image-distortion compensator which amends distortion of an image from the image 
pick-up data obtained with image pick-up equipment, and memorizes the correction 
function for amending image pick-up data, An amendment means to amend image 
pick-up data using the correction function read from this correction function storage 
section is included. The above-mentioned correction function storage section The 
location of the thing near the center in the image pick-up data in the focus which 
extracted the focus and was obtained from the image pick-up data of the pattern for 
proofreading whose arrangement of the focus is known, Based on the data about the 
pattern for proofreading, the mapping function which assumed image pick-up 
equipment to be a pinhole camera is computed. It is mostly characterized by 
memorizing the correction function obtained based on the criteria location of the 
focus of image pick-up data which computed the criteria location in the image pick-up 
data of the focus all over the districts, and was obtained, and the location of the focus 
in image data using the obtained mapping function. 
[0011] 

[Function] Thus, according to this invention, by the comparison of the focus 
arrangement on the screen extracted from the monotonous image pick-up data 
describing the proofreading pattern with which the array of the focus has become 
settled, and the location in an actual screen, when a camera is assumed to be a 
pinhole camera, a mapping function is obtained. Since it is asking for the mapping 
function by this technique, the plate describing a proofreading pattern can be applied 
also in the location of arbitration, and it is not necessary to search for a camera and 
monotonous physical relationship with other means. 

[001 2] By this technique, it asks for a mapping function only using the whole screen or 
some data, and it can amend so that it may be suited. However, if the image 
amendment before of an image and after amendment uses the mapping function which 
is well in agreement, there will be few amounts of amendments of a location, the 
deficit in the flash from a screen and the side edge by shrinkage will decrease, and 



desirable effectiveness will be acquired. Then, by this technique, the data near the 
center of a screen have a small distortion in many cases, and a mapping function is 
obtained using the data near a center using the property which is well in agreement 
with the image obtained from the pinhole camera. 

[0013] Next, the criteria-location of the focus of the whole screen is computed using 
this mapping function. And a correction function is computed from the comparison of 
this criteria location and the location on an actual screen. For this reason, compared 
with the case where distortion of an image is amended, a suitable correction function 
is only computable from the location on a screen. And suitable image data can be 
amended using this correction function. 

[0014] Furthermore, according to this invention, in the case of detection of the focus, 
the boundary line of a pattern is detected from the change condition of not only the 
processing for every pixel but pixel data, and the focus is detected according to the 
condition of this boundary line. For this reason, location detection of the exact focus 
can be performed compared with the processing for every mere pixel. 
[0015] moreover — thus, since image pick-up data are amended using the obtained 
correction function, an image without distortion can be obtained, according to the 
image processing, suitable location detection etc. can be performed and the precision 
of measurement of the environment condition in a robot etc. can be improved. 
[0016] 

[Example] Hereafter, the example of this invention is explained based on a drawing. 
[0017] Drawing 1 is the block diagram showing the whole location detection equipment 
configuration concerning this invention. A picture input device 10 consists of CCD 
cameras etc., changes into a digital signal the analog video signal acquired as a 
luminance signal for every pixel by the A/D converter, and outputs it. the data from a 
picture input device 10 are supplied to the input image data storage memory 12 
having — the usual case — one frame (one screen) — it memorizes here. 
[0018] And the image data read from here is supplied to image data correction 
equipment 16 through a switch 14. This image data correction equipment 16 has 
memorized the correction function beforehand, amends the image data for every pixel 
supplied from the input image data storage memory 12 using a correction function, 
and supplies this to the output image data storage memory 18. This output image data 
storage memory 18 reads the data which memorize the image data for one frame and 
are memorized here one by one, and supplies them to the image output unit 20. This 
image output unit 20 consists of CRT, and performs the display according to the 
image data memorized by the output image data storage memory 18. In the usual case, 
the data memorized by the output image data storage memory 18 are read 
synchronizing with a Vertical Synchronizing signal and a Horizontal Synchronizing 
signal, and through a D/A converter, it changes into a predetermined video signal and 
outputs to CRT. The display by which distortion of an image was amended in the 



image output unit 20 by this is performed. 

[0019] Moreover, the image-processing section 22 is also connected to image data 
correction equipment 16. This image-processing section 22 performs various kinds of 
image processings (for example, pattern recognition) etc. based on the image data 
amended by image data correction equipment 16, and performs the shape recognition 
of the object in image data, location measurement to that object, etc. Here, this 
location measurement is good to prepare two picture input devices 10 and to detect 
by trigonometry etc. from processing of the input image data from two picture input 
devices 10. 

[0020] On the other hand, focus detection equipment 24 is connected to the switch 
14. Then, the image data stored in the input image data storage memory 12 can be 
supplied also to focus detection equipment 24 by switching a switch 14. That is, in 
performing the usual image processing and the output of an image, in case it supplies 
input image data to image data correction equipment 16 and computes the correction 
function in image data correction equipment 16 (at for example, the time of 
initialization), a switch 14 is switched so that input image data may be supplied to 
focus detection equipment 24. 

[0021] Focus detection equipment 24 processes the input image data supplied, for 
example, detects the location of the focus, such as a checkered intersection. And this 
detection result is supplied to the focus location storing memory 26. In the usual case, 
the input image data for one frame is processed, and the data about the location of 
the focus extracted out of this are stored in the focus location storing memory 26. 
[0022] Next, the data about the location of the focus for which the focus location 
storing memory 26 was asked from the input image are supplied to the mapping 
function listing device 28. This mapping function listing device 28 assumes that it is 
that to which a picture input device 10 operates as a pinhole camera, and the mapping 
function of a picture input device 10 is computed from the data inputted. And the 
mapping function detected by the mapping function listing device 28 shifts, and is 
supplied to detection equipment 30. To this gap detection equipment 30, the data 
about a focus location are also supplied from the focus location storing memory 26, 
and the comparison of the criteria location of the focus obtained using the mapping 
function and the location of the detected focus detects both gap to it. The correction 
function listing device 32 is connected to this gap detection equipment 30, and from 
the gap supplied, this correction function listing device 32 computes the correction 
function for amendment to the data obtained by the mapping function, and supplies 
this correction function to the correction function storing memory 34. 
[0023] Then, image data correction equipment 16 can read the correction function 
memorized by the correction function storing memory 34, and can supply the image 
data which amended input image data and was amended by this in the right location at 
the output image data storage memory 18 and the image-processing section 22. 



[0024] Drawing 2 is a flow chart which shows the actuation by the whole correction 
function calculation of this example. Thus, in this equipment, the focus is extracted by 
processing input image data in focus detection equipment 24 (S1). Next, the location 
on the pattern of the obtained focus is determined (S2). And based on the data about 
the focus with which the location on a pattern was determined, and the photoed data 
about the pattern for a configuration, the correction function for detection of the gap 
in the data processed and obtained with creation of a mapping function and a mapping 
function and the gap dissolution based on these detection values is created, and a 
correction function is determined (S3). 

[0025] The flow chart for explaining actuation of a focus extract to extract drawing 3 
of the focus is shown. First, in this example, for correction function calculation, a 
specific proofreading pattern is prepared and a correction function is computed by 
image data processing about this proofreading pattern. Then, detection of the focus in 
a proofreading pattern performs detection of this focus. 

[0026] In this example, a checkered pattern as shown in drawing 4 as a specific 
pattern is used, and let the intersection of this checkered pattern be the focus. Then, 
first, a template is hit to image data and the map of the value is made (S1 1). Here, the 
data about the circumference pixel of 36 including the processing-object point shown 
by the round mark as indicates to be a template to drawing 5 are processed, and the 
value of the point is calculated. In this example, the value about that pixel is added 
about the data of the nine upper left, and the data of the nine lower right, and the data 
about that pixel are subtracted about the data of the nine upper right, and the data of 
the nine lower left. By performing such processing, the value of the data processed by 
the template turns into maximum or the minimum value in the focus (intersection) of a 
checkered proofreading pattern. Then, the value for each point is calculated about all 
the data for one frame using this template. In addition, in here, the value about each 
point takes the absolute value. Then, each intersection will take the maximal value. 
[0027] Next, the nearby maximal value is taken out sequentially from the map obtained 
by doing in this way (S12), and the location by Rhine fitting is computed to the point 
(S13). And the focus location computed by this Rhine fitting is outputted (S14). And 
S12-S13 are repeated to the last maximum point (S15). 

[0028] Next, Rhine fitting in S13 is explained. If the focus is computed by processing 
using a template, this location will serve as a point of the one maximal value. And when 
the above templates are used, the actual focus should take the lead in the square 
formed at the maximum point, and this right, the lower right and three lower points. 
However, in actual measurement, a proofreading pattern does not restrict being 
arranged to a camera (picture input device 10) at the right angle, and a square core is 
not necessarily a right focus location in it. 

[0029] Then, in this example, as shown in drawing 6 (A) - drawing 6 (C), the coordinate 
of the focus is searched for from the change condition of the surrounding image data 



of the called-for focus. For example, binarization of the image data is carried out to 0 
or 1 with a predetermined threshold, and it asks for a bordering straight-line type from 
the change condition of this binarization data. A vertical straight-line type can 
calculate the absolute value of the differential value which can be set horizontally, as 
shown in drawing 6 (B), it can search for the point that a value is big, one by one, and 
can search for it by the formula of the straight line which passes along these points. 
Moreover, a horizontal straight-line type can differentiate the value for every vertical 
pixel, as shown in drawing 6 (C), and it can ask for the formula of the straight line 
which passes along the point that the absolute value of a differential value is big. And 
a focus coordinate can be searched for by asking for the intersection of two obtained 
straight-line types. For example, according to the example of drawing 6 , the 
coordinates of the focus for which it asked from the template are (5.5, 5.5), but the 
horizontal straight line for which it asked as mentioned above is 2X+10Y-65=0, and a 
vertical straight line is 10X+2Y-69=0. Then, the intersection of these two straight 
lines is set to (35 / 6= 5.833, 32/6=5.333), and can ask for a more exact focus location. 
[0030] Thus, with the coordinate of the focus called for only from the data only 
obtained by the template, the exact focus in consideration of the inclination of the 
proofreading pattern which is not called for etc. is detectable. 

[0031] In order to determine not the location detected on image pick-up equipment 
(on a screen) but the location used as the criteria on count about the location, next 
the focus called for as mentioned above on the pattern of the focus, the location on 
the pattern (location on the array corresponding to a proofreading pattern) is 
determined. That is, a procedure as shown in drawing 7 determines the location on the 
pattern of the focus. In addition, the coordinate on (X, Y), and a pattern is expressed 
with (m, n) for the coordinate on a screen. 

[0032] First, the obtained focus is sorted in the order near the core of a screen (S61). 
And the four directions of the central point are determined in the following logic about 
the sorted focus (S62). 

[0033] That is, the four directions of the central point are determined according to a 
flow chart as shown in drawing 8 . First, the first point P0 (X0 and Y0) which is a point 
nearest to the core of a screen is taken out, and this point is set as =(m, n) (0 0) = 
core (S21). Next, the point P1 (X1 and YD near that degree is taken out at the core, 
and let this point be =(m, n) (1 -0) = left (S22). Next, the point P2 (X2 and Y2) near a 
core is taken out to that degree (S23), and it judges whether this point is the right 
(S24). The following operations of the coordinate of a main point and the coordinate of 
a left point perform the judgment of whether to be this right. 

[0034] X2 - =2X0-X1 Y2 -=2Y0-Y1 [ and ] — by such judgment, when P2 (X2 and Y2) 
is the right, let this point P2 (X2 and Y2) be the P2 = (1 0) right (S25). And the point P3 
(X3 and Y3) near a core next is made into a P3 = (0 -1) top (S26), and the following 
point P4 (X4 and Y4) is made into the bottom of =(m, n) (0 1) = (S27). 



[0035] On the other hand, in S24, when P2 (X2 and Y2) is not the right, the point P2 
(X2 and Y2) is made into =(m, n) (0 -1) = top (S28). And the point P3 (X3 and Y3) near 
a core next is taken out (S29), and it judges whether this point is the right (S30). The 
judgment of whether to be this right is that point P3 like an above-mentioned case. 
Left P2 And core P0 Whether it has an above-mentioned relation performs. And it 
sets to S30 and is P3. When judged with it being the right, a point P3 (X3 and Y3) is 
made into =(m, n) (1 0) = right (S31), and the following point P4 (X4 and Y4) is made 
into the bottom of =(m, n) (0 1) = (S32). 

[0036] Moreover, when a point P3 (X3 and Y3) is not the right in S30, make a point P3 
(X3 and Y3) into the bottom of =(m, n) (0 1) = (S33), and let the following point P4 (X4 
and Y4) be =(m, n) (1 0) = right (S34). 

[0037] Thus, let the point nearest to a core be the left first (S22). And when there is 
no point (the 2nd) near a core next in the opposite side (right-hand side) of the point 
made into the left, that point is made into a top (S28), and the point (the 3rd) near that 
degree judges whether it is the right, and if it is not the right, make this point into the 
bottom (S33), and let the point (the 4th) near that degree be the right (S34). On the 
other hand, if the point near the 3rd is the right, this will be made into the right (S31) 
and the point near the 4th will be made into the bottom (S32). Moreover, if the point 
near the 2nd is the right, this will be made into the right (S25), the point (the 3rd) near 
the degree will be made into a top (S26), and the 4th will be made into the bottom 
(S27). Such processing determines the location on the pattern of the point of the four 
directions of the surrounding focus of the central point. 

[0038] Thus, when the location on the pattern of the point of the four directions of the 
central point is determined, the location on the pattern of each point is determined 
next (S63). This is performed in the following logic. In addition, it sets to the following 
explanation and is Subscript c. A core and r The right and I For the left and a top, the 
bottom is [ u and ] d. It is shown. 

[0039] That is, a procedure as shown in drawing 9 and drawing 10 performs. Although 
this processing is performed about all the focus, one-point Pc = (m, n) (mc and nc) 
nearest to a core is taken out out of the point which must be processed first (S41). In 
the case of the first loop formation, it is this point Pc. It becomes core (m, n) = (0 0). 
And this point Pc It considers as criteria and the point of Hidari of this point is 
determined as follows (S42). Namely, point Pc It judges whether it receives, and 
Hidari's point PI = (m, n) (mc-1 and nc) does not exist, and right-hand side point Pr = 
(m, n) (mc+1 and nc) exists (S421). In corresponding to this condition, it looks for a 
point with the coordinate near (2Xc-Xr and 2Yc-Yr) on that screen out of the 
extracted focus (S422). Namely, point Pc taken out first Pr which already exists The 
coordinate on the screen of a point to point Pc A left-hand side point is looked for. 
And when the point of this left-hand side is found, let (S423) and its point be PI =(m, n) 
(mc-1 and nc) = left (S424). Moreover, when the point which corresponds in S423 



when there is nothing that corresponds in S421 is not found, it moves to the following 
activation of S43 as it is. 

[0040] And according to the same procedure as these S42, a Pr =(m, n) (mc+1, nc) = 
right and Pu =(m, n) (mc, nc-1) = top and the bottom of Pd =(m, n) (mc, nc+1) = are 
determined (S43, S44, S45). Next, upper left point PuKm. n) = (mc-1, nc-1) is 
determined (S46). For this reason, it judges whether Pul(m, n) = (mc-1, nc-1) exists 
first (S461). And when this point Pul does not exist, it judges whether left-hand side 
point PI = (m, n) (mc-1 and nc) or upper point Pu = (m, n) (mc, nc-1) exists (S462). 
And these two points PI And Pu When both exist and it is set to NO in S462, a point 
with the coordinate near (XI+Xu-2Xc and YI+Yu-2Yc) on a screen is looked for out of 
the focus (S463). That is, the point of the upper left is looked for from three known 
points. 

[0041] And when the point in this coordinate is found, let (S464) and its point be the 
PuKm, n) =(mc-1, nc-1) = upper left (S465). Here, when the upper left point Pul 
already exists in S461, or when there is no point which corresponds in S464, it moves 
to the following activation of S47. 

[0042] And the same procedure determines the PdKrn, n) =(mc-1, nc+1) = lower left, 
the Pur(mc+1, nc-1) = upper right, and the PdKrn, n) =(mc+1, nc+1) = lower right (S47, 
S48, S49). And the point made into the processing object judges whether it is the last 
point (S50), and repeats such processing to the last point. 

[0043] Thus, in relation with the point already determined one by one, the relative 
position is determined from the point near a core. 

[0044] Thus, by above-mentioned logic, the location on a pattern can be determined 
about each point, and a 2-dimensional coordinate (m, n) is given about these. 
[0045] When determining the location on a pattern as mentioned above, it does not 
become considering the point of the location used as a checkered intersection as the 
focus. And when the pattern location of the focus extracted by the above-mentioned 
procedure is determined, a location is not determined about the focus as which the 
location was determined, and the point which is not in specific relation. Although the 
extract was carried out as the focus, this is the point which must not exist and 
removes such the isolated point from the data of a specific pattern (S64). 
[0046] Thus, about each focus, the location (X, Y) and pattern location (m, n) on a 
screen are determined, and this is memorized by the focus location storing memory 26. 
[0047] As it is the decision **** of a correction function, when the location on the 
pattern (m, n) is altogether determined about the focus obtained from image data, it 
asks for a correction function from the location of these focus, and the configuration 
of a specific pattern. That is, a procedure as shown in drawing 11 determines a 
correction function. First, it asks for a mapping function from the pattern location (m, 
n) of the focus where the relative location called for as mentioned above was 
determined, and the concrete location on a screen (X, Y) (S51). Here, although it asks 



for this mapping function, not all the focus uses, but it uses only the focus of 1 /4 near 
a core for it. This is because it is thought that there is little nonlinear distortion etc. 
and it can ask for the mapping function near a subject-copy image about the point 



near a core. 



[0048] Next, the location which serves as criteria about the focus asked for the 
location on all patterns is computed using the called-for mapping function (S52). Next, 
the criteria location called for by the mapping function is compared with the locations 
X and Y on an actual screen, and it asks for both gap, and asks for the correction 
function for amending this gap (S53). And the called-for correction function is 
outputted to the correction function storing memory 34, and it memorizes here (S54). 
[0049] calculation of a mapping function ~ here, calculation of a mapping function is 
explained. The actual photography in a camera can be ideally approximated to 
photography by the pinhole camera. That is, as shown in drawing 12 , the beam of light 
from one point [ of three-dimensions space ] P is projected on an image pick-up side 
(flat surface) through a focus. And the image on an image pick-up side is memorized 
by the input image data storage memory 12 through photo electric conversion and an 
AD translation. And the location on this memory 12 (X, Y) turns into a location on a 
screen. And according to the pinhole-camera model, it is known that there are 
[ relation of the following linear functions (mapping function) between X and Y ] a 
location on a pattern (m, n) and a location on a screen. 
[0050] 

X=(cm+dn+e)/(am+bn+1 ) 
Y=(fm+gn+h)/(am+bn+1 ) 

Therefore, about the focus of a large number extracted as mentioned above, the 
location on this pattern (m, n) and the location on a screen (X, Y) are substituted, and 
each constant a-h is calculated with the least square method. The mapping function 
at the time of assuming the camera in this system to be a pinhole camera by this is 
determined. In addition, this least square method is shown, for example in the Iwanami 
Shoten issue "Iwanami lecture information-science 18 numerical calculation (the 
January 8, Showa 57 1 st ♦*****)" etc. 

[0051] And the focus used in order to ask for this mapping function makes only a near 
thing (the focus of the whole quadrant) from a core. The part with this near a core is 
because it is considered to be the right to approximate with a pinhole camera. Then, it 
asks by correlation with each high multiplier from the focus such near the core, and a 
mapping function with a high precision is called for. 

[0052] Since the degree which moves a relative position to a criteria location was 
asked for the mapping function as mentioned above, the location which used this 
mapping function about all the extracted focus is computed. That is, when a picture 
input device 10 is a pinhole camera, the location on the count in which each focus as 
which the location on a pattern (m, n) was determined should be located is determined 



by the above-mentioned mapping function. Then, a criteria-location (x y) is computed 
using a mapping function about all the focus as which the location on the extracted 
pattern (m, n) was determined. 

[0053] Calculation of a correction function, next the criteria location (x y) called for by 
doing in this way and the location on an actual screen (X, Y) are compared, both gap is 
detected, and it asks for the correction function for canceling this gap. This correction 
function is performed by determining each multiplier with the least square method 
using the following polynomials. 
[0054] 

x=a1 X4+b1 X3 Y+d X2 Y2+d1 XY3+e1 Y4 + f1 X3+g1 X2 Y+h1 XY2 + i1 Y3+J1 X2+k1 
XY+11 Y2+m1 X+n1 Y+o1 y=a2 X4+b2 X3 Y+c 2X2Y2 + d2 XY3+e2 Y4+f2 X3+g2 X2 
Y+h2 XY2+i2 Y3+j2 X2+k2 XY+I2 Y2+m2 X+n2 Y+o2 Thus In this example, a criteria 
location (x y) is computed about all the focus on a screen using the mapping function 
which asked for the camera from the focus near the center of a screen the bottom 
wholly as a pinhole camera, and was called for in the mapping function of a case. And 
a correction function is computed by the comparison with the computed criteria 
location (x y) and the location on an actual screen (X, Y). Therefore, it can ask for a 
very exact correction function compared with the case where it is going to amend 
distortion of an image as it is. 

[0055] Moreover, each focus will be outputted to a location when a picture input 
device 10 operates as a pinhole camera by memorizing this correction function in the 
correction function storing memory 34, amending the image data supplied from the 
input image data storage memory 12 using a correction function, and displaying on the 
image output unit 20. Therefore, when displaying various images, an image without 
distortion picturized with the pinhole camera can be outputted to the image output 
unit 20. 

[0056] Furthermore, that a picture input device 10 can be operated as a pinhole 
camera can grasp correctly the location of the photoed object actual from the 
location of each image when image data is processed by the image-processing section 
22, and when a stereo image etc. is used, it can perform the processing 
measuring-point measurement correctly. 
[0057] 

[Effect of the Invention] As explained above, according to this invention, the focus is 
extracted from the proofreading pattern drawn on monotonous, and a mapping 
function is created from the location on the screen of the focus near a photograph 
center. And the ideal location of the focus is computed about the focus of the whole 
screen using this mapping function, and a correction function is computed from the 
comparison with this criteria location and the location of the focus of the whole 
screen. Therefore, it becomes measurable, without measuring installation or a location 
for a plate correctly, and pinpointing the location of the pattern, and a correction 



function will become very exact, and image data can be amended using this. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 1] It is the block diagram showing the whole example configuration. 

[Drawing 2] It is a flow chart explaining whole actuation. 

[Drawing 3] It is a flow chart explaining focus extract actuation. 

[Drawing 4] It is the explanatory view of a checkered proofreading pattern. 

[Drawing 5] It is the explanatory view showing the configuration of a template. 

[Drawing 6] It is drawing explaining actuation of Rhine fitting. 

[Drawing 7] It is the flow chart of the actuation for determining the relative value of 
the focus. 

[Drawing 8] It is the flow chart which shows the procedure of determining the four 
directions of the focus. 

[Drawing 9] It is the flow chart which shows the procedure of determining the relative 
position of the focus. 

[Drawing 10] It is the flow chart which shows the procedure of determining the 
relative position of the focus. 

[Drawing 11] It is a flow chart explaining actuation of correction function decision. 

[Drawing 12] It is an explanatory view explaining the principle of a pinhole camera. 

[Description of Notations] 

10 Picture Input Device 

12 Input Image Data Storage Memory 

16 Image Data Correction Equipment 

18 Output Image Data Storage Memory 

20 Image Output Unit 

22 Image-Processing Section 

24 Focus Detection Equipment 

26 Focus Location Storing Memory 

28 Mapping Function Listing Device 

30 Gap Detection Equipment 



32 Correction Function Listing Device 
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(m, n) = (m c -1, n c ) ^OfefflJ 
€>*P r (m, n) = (m c +1, n c ) ^SSfSfr 
5fr£W&-fi> (S 4 2 l) o c:©*fNcRiIif*«^ 

(2X C -X r . 2 Y c -Y r ) KifiV^*«f (S 4 
2 2) o T&*>^> SSJfc|5lt>ffiLfcj&Pc 

t§p r ^©nfiiifn±»^A>?,^p c <ofe«eicDjs*s 

"To fit, CcO&ffllO^A^O^ofc^-a-tCti (S4 
2 3) , ^(D^^Pj (m, n) = (m c - 1 , n c ) 
= fet:-rS (S 4 2 4) o Sfc, S 4 2 1 KtS^XWi^ 
f^feO^'^l^ S 4 2 3tfe^tISt5Mli 

[0 0 4 0] ^LT, dOS 4 2 i:|Bl1i(D¥llEtC^V\ 
P r (m, n) = (m c +1, n c ) =£> P u (m, 
n) = (m c , n c -1) =±> P d (m, n) = (m 
c . n c + 1 ) =T*tt^r* (S43, S44, S4 
5) o #fC^ fe±«0^P u i (m, n) = (m c -1. n 

c - 1 ) s (s 4 6) o cvrctbic sfp u i 

(m, n) = (m c -1. n c -1) tf#ft-f3fr§fr 
(S4 6 1) o ^LT, LO,SP,„tfSftL 
^V^-a-tcti, fefUJO^Px (m, n) = (m c -1, 
n c ) %.fclA±m<D&P u (m, n) = (m c . n c - 

i ) ^ft-rs^SA^^js-rs (s 4 6 2) . 

T, CliOZOiD^P, WP,, OWM^SU S4 6 



±<Dmm# (Xj +X U -2X C , Y t +Y U -2 

y c ) tcj£i/^*«rr cs 4 6 3) o -r^fr-s, K*q« 

3 j^A"> P. *<D&±<D&1tm?o 

[0 0 4 1 ] ZLX, dOffitSlCfe^^mo^ofcil 
•g-tCii (S 4 6 4) , ^(Dj&^rPu! (m, n) = (m c 
- 1 , n c - 1) = £±i:t5 (S 4 6 5) . CCT. 

S 4 6 1 tc*3^rffifcfe±o^p u i^#ffibrc«-&, * 
fctiS 4 6 4 ^fe^Tisa-r^>*^^^o/cti^-^fe^ 

Tti> *<0S 4 7 (D^filC&Z> 0 
[0 0 4 2] ^LT, |B]«(D#|li(Cj;-DTP dl (m, 
n) = (m c -1, n c +1) =£T> P ur (m c + 
1, n c -1) =£±> P dr <m. n) = (m c +1, 
n c + 1) = 2rF*i*jrf 3 (S47, S48, S4 
9) „ fit, MJItttifc Lfcj«3^««<0jS*PS* , * ! PJ 

(S50), coiasM^Mo^s-e*^ 

"To 

[0 0 4 3] ClO^tcLT, cp)L4c5£iv^^e>]fi^K 

[0 0 4 4] COi^tCLT, iijZEcDf&SiCitK 

iblZ^^X^RfxAW. (m, n) *^x.?)ns„ 
[0 0 4 5] ±ft<D£5iCLX'^ — y±.<D{iLW%:'ik'xL 

tzt>\ tfife/^Z — XDr— $tfrtb&&-DXli.%:Z>%:^& 
X&0, C<D£5&WiL&%ffi£-?% (S6 4) „ 

[0046] ccD^oicLx, &tmmz~3^x. mm 

±<DtiLW (X, Y) jSkW^—yitLM (m, n) 
[0 0 4 7] *HIE!3B»<Pifee 

\ J ^X±X^<D^^ — >±(DiiLW. (m, n) £r?£^LfciJ§ 

A^MiEKa^^^^o eh n«:^-r«fc'5* 

ttB Cm. n) £:Hffi±&c:fctt3*#:iW&{4B (X, 
Y) i:^5>^#SI»**«>* (S5D . 

3fiV^ 1 /4 (D^jSO^^rffl^Sc cn^i, tp^fCjfiV^ 
[0 0 4 8] ^(C> *46e.tlfc^#K3&*fflV\ 
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^fiB^ffffi-rs (s 5 2) o ^{tMtsac^D* 

^T5l<465 (S5 3) o ^LT, ^£D^#>btl.fc«IEIiii5( 
£ffiIEIIBI8rt&»**»J 3 4lctiit)L<l£lcmm?% (S 
5 4) o 

[0049] mim&QMm 

tt«ms©»Kfi, Vyfr—frii *=y\z&% 

SH&teifittn***. mi 2K^.?&o\c = 

{cIS'B^nSo ?LT, <1©^*U 1 2±fc*3ttSffi« 
(X, Y) fcfcBffi±cDfifi£&5o fit, t?y*-'i> 
#^v*-r'Wc«fcft{i\ — >±OjfiH (m, n) i: 

psiifiiiottia^ x , y t mmc it. a<d «t 5 #rai& 

[0 0 5 0] 

X= (cm+dn + e) / (am+bn+1) 
Y= (fm+gn + h) / (am+b n + 1) 

t,"»T\ C©/<£— y±.<D\i!M (m, n) &ttWlE±©ft£ 
B (X, Y) «UL, =&S»a~h*a/jomiSfc«t 

x = al X 4 +bl X3 Y+cl X 2 
+ g 1 X 2 Y + hl XY 2 + il Y 3 + 
+ nl Y + ol 

y = a2 X 4 +b2 X 3 Y+c2 X 2 
+ g2 X 2 Y + h2 XY Z + i2 Y 3 + 
+ n2 Y + o2 

— /l/*^7fcLT*i8:brc«^<0^«IIB»*iiiDB©f * 

±©£T©^&tCOV>TSWB (x, y) fcffWT 

5o fLt, stb^n/ta^fiH (x, y ) t, mm<D 
iM±©tiH (x, y) t<Dttmc&<9, tmmmzw 
m-r^o for, H«<o^** j e<Dssffi]Eu«t'5 t-r 

[0 0 5 5] *fe, (lOffiiEgi^MiEM^tS^^^'J 
3 4(CfB«LTfc^, A^H^-^tSiW^^U 1 2fr 
6«l&*n*H«7 f -***liEB8»*ffli/'>TffiiELlii« 
m^8g2 0t^t5ttf;:<t-3T 1 BfS!A±j£§Bl 



rg&isjs t*fgf4^ 1 8 mmm om 57^ 

1 £ 8 Bfg 1 M«fT) J frUlc^ZnT^Z, 

[0 0 5 1 ] fit, c©¥«BIII»**46«fcJ6fc:fflv^ 

EMSfoS) i:-T«o ens, ttJi^fciav^ti, 

[0052] ffla^a^g^aMi^Mfej: a 

^BWr?.o T&ta5, BffA^SBi O^tfy^-^ 
A^5T?*o^Ji-a-{C{±, /^->±OfiB (m. n) 

^-y±<DfsiB (m, n) *^3!^nrc-r^T©#^ 

fcOVT, S^Wfifi (x, y) *?«M»%ffl^T)f 

[0053] MffiMl^Mffl 

CCD«tdfcLT*4i)P)tl/-c:M^i4B (x, y) t 

nm?>mm±(Dim (x, y) *it«u mmo-m* 

/hri3ltj£tC £ -3 T rfcaTT 5 c £ o Tff 9 0 
[0 0 5 4] 
Y 2 +dl X Y 3 +el Y 4 + f 1 X 3 
j 1 X 2 +kl X Y+ 1 1 Y 2 +ml X 

Y 2 +d2 XY 3 +e2 Y 4 +f2 X 3 
j2 X 2 +k2 XY+12 Y 2 +m2 X 

[0056] $p>ic wmxtsmnx o£e>*-;i/# 

£QJIg[5 2 2 o Tltf- £ L fc^fc, &B 
[0 0 5 7] 

[fgB^o?i)S] u±ikwl rebate, *^^fc«tntf, 

5„ ^LT, CO^MiS(^ffl^T^M^oa$iWffiB 
?rBffi^:f*0#S!(iStcoi/^TS;mL, COlffili:! 

£o fcT, ^^iEaicSBt. L < lifeB^IEttfcH- 
SiJ LT?®/ ^ ^— > OfiB^r^'f 3 C i: ^ < I+S'J A 55 "J 
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[03] wabsam«ff*«wr*7o-f-y-h"Tf* 
[04] <$mmmm^z->(Dmwmv&z,o 

[0 5] x^^^-ho^^-TSiWHT^^o 
[06] 7^V7-f 'yf-O^ftMliWrSiT^ 

[0 7] «r*^©ffi^%^-r«/-ci6©«i^<07n- 
[08] W«£©±T££*&£'rs#MB**'r7n- 

[0 9 ] ««£©ffl*MM*Rj&r § ^m^mty n - 
[010] ^ajswffiWttH^st^-rs^jo^^-r^n 



[0 11] ffiIEMiS(a5£0»jf / F^irat-5 7P-^ + - 

[0 1 2] Vf>!h-;l/*^7©Iga*SiM-rstttW0t? 

1 0 BffsA^SB 

l 2 A^SHt^-^fStt^U 

1 6 BffiU'r-^ffiiESH 

1 8 m#B»r-**»«**y 

2 o mfsm^^g 

2 2 Hf^MiigfS 

2 4 #iit^^asB 

2 6 ita^ffiB^iw^^'j 

2 8 mmwLif^mm 

3 o -rn^wsB 

3 2 WiiElMtJicSK 



[02] 



-S1 



S2 



S3 



[04] 




[0 3] 
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[05] 
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- - - ♦ * 
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(A) 



10X.2Y-69=0 
(B) 



(C) 



[0 7] 



[0 11] 



S61 



552 



~S6f 



S 5 1 



S 5 2 



S 5 3 



S 54 
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[08] 



■(X,Y) 
: (m. n) 



P Q (0, 0) <iM» t+z 



S21 



p J c~i, o) cso t-a-s 



~S22 



;K*£,P2(X 2 ,Y2) tut 



kS23 




J*r/P2(X 2 ,Y 2 )t 

P 2 IU.-1) iys C±) 



-528 



S25 



^P2(X2,Y 2 )J 
P 2 (1. 0) i-i-* «j) 



:jCA#»P 3 (X3,Y3te 



S29 



S26- 



^^,P 3 (X3,Y3)^ 
Pj (O, -1> (-h) 



.£vP 3 (X 3 ,Y 3 )S 

Pg (0, 1) (T) 




I 



p 4 (i. 0) t-te, <«) 



S33 
■ S34 







£,P 3 (X 3 Y 3 ) 
p 3 ci, 0) tti cfi) 



*»<&.a(x 4 ,\u;sr 

P 4 (O. 1) i-f-s (T) 



;5C **,P 4 (X 4 ,YJ* 

P 4 (0, 1) CT) 



■S31 



S32 
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[09] 



© 



541 



NO 




S421 



YES 



£,<2Xc -X r ,2Y c -Yr)<;&" 
n't 



SA22 



S42 




* P j (m c -l. n c ) 



R|«|sP r (m B + 1. n c ) 



fflttl=P u (m c . n c - 1) 

<±> &wmtz 



I 



-SA23 



SA24 



-SA3 



S45 



(13) 



5-274426 



[010] 



YES 




YES^ ^ p i t,n c _1 - " c > 



(X, + X u ~'JX a , Y| +Y U -2Y C ) 



S461 



SA62 



^S463 





YES) 





















little P d| (m c -1, n c + 1) 

i zz: 



l"J#ICP up Cm c +1. n Q -1J 



imtifcP^. (m c +1. n e + 1) 



■S464 
SA65 

S48 
S49 



SA6 




S50 
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[0 1 2] 
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